Airborne reconnaissance systems have been widely used for many years now, particularly for obtaining images from the air of areas of interest.
Originally, a film camera was used on board aircraft for capturing the images. The main problem of the airborne, film-camera based reconnaissance system is the length of time required for developing the film, an operation that can be performed only after landing. This problem has been overcome in more modern systems by the use of a one-dimensional vector or a two-dimensional array of light-sensitive sensors in the camera for obtaining electronic images that are then electronically stored within the aircraft, and/or transmitted to a ground base station. This is generally done in such systems by scanning by the light-sensitive sensors of the area of interest in the direction of the flight.
Airborne reconnaissance systems are generally used to obtain images of hostile areas, and therefore the task of obtaining such images involves some particular requirements, such as:    1. Flying the aircraft at high elevations and speeds in order to reduce the risk of being targeted by enemy weapons, and in order to widen the area captured by each image;    2. Trying to capture as much relevant image information as possible during as short as possible flight;    3. Trying to operate under various visibility conditions, while not compromising the resolution of the images and their quality.    4. Trying to photograph rough terrains (e.g., high mountains, areas having sharp ground variations), in high resolution and image quality.
The need for securing the reconnaissance aircraft, while flying above or close to hostile areas has significantly increased flying costs and risks, as sometimes the reconnaissance mission requires escorting of the aircraft by other, fighter aircrafts. Therefore, the need for enabling a short and reliable mission is of a very high importance.
There are several other problems generally involved in carrying out airborne reconnaissance. For example, capturing images from a fast-moving aircraft introduces the need for the so-called Forward Motion Compensation (Hereinafter, the term “Forward Motion Compensation” will be shortly referred to as FMC. Motion Compensation in general will be referred to as MC), to compensate for aircraft movement during the opening of the camera shutter (whether mechanical or electronic; in the latter case, the opening of the camera shutter for the purpose of exposure is equivalent to the integration of light photons by the light-sensitive components).
When light-sensitive sensors are used in the camera (hereinafter, this type of image capturing will be referred to as “electronic capturing” in contrast to “film capturing”, wherein a film-type camera is used), three major scanning types are used:    i. The Along-Track Scanning (also known as “push-broom scanning”)—In a first configuration of the Along-Track Scanning, the light-sensitive sensors are arranged in a one-dimensional vector (row), perpendicular to the flight direction. The scanning of the imaged area is obtained by the progression of the aircraft. In one specific configuration of Along-Track Scanning, generally called Along-Track TDI (Time Delayed Integration) configuration, a plurality of such parallel one-dimensional vectors (pixel-rows) perpendicular to the flight direction are provided at the front of the camera forming a two-dimensional array. In that case, however, the first row of the array captures an area section, while all the subsequent rows are used to capture the same section, but at a delay dominated by the aircraft progression. Then, for each row of pixels, a plurality of corresponding pixels of all the rows in the array, as separately measured, are first added, and then averaged in order to determine the pixel measured light intensity value. More particularly, each pixel in the image is measured N times (N being the number of rows) and then averaged. This Along-Track TDI configuration is found to improve the signal-to-noise ratio, and to improve the image quality and the reliability of measuring.    ii. The Across-Track Scanning (also known as “Whiskbroom Scanning”)—In the Across-Track Scanning, a one-dimensional sensing vector of light-sensitive sensors, arranged parallel to the flight direction, is used. The sensing vector is positioned on gimbals having one degree of freedom, which, during the flight, repeatedly moves the whole vector right and left in a direction perpendicular to the direction of flight, while always keeping the vector in an orientation parallel to the direction of flight. Another Across-Track Scanning configuration uses a moving mirror or prism to sweep the line of sight (hereinafter, LOS) of a fixed vector of sensors across-track, instead of moving the vector itself. In such a case, the Across-Track Scanning of the area by the gimbal having one degree of freedom, while maintaining the forward movement of the aircraft, widens the captured area. Another configuration of the Across-Track Scanning is the Across-Track TDI configuration. In this configuration there exists a plurality of vectors (columns) in a direction parallel to the flight direction, forming a two-dimensional array. This Across-Track TDI, in similarity to the Along-Track Scanning TDI, provides an improved reliability in the measuring of pixel values, more particularly, an improvement in the signal-to-noise ratio.    iii. Digital Framing Scanning: In Digital Framing Scanning, a two-dimensional array of light-sensitive sensors is positioned with respect to the scenery. In U.S. Pat. No. 5,155,597 and U.S. Pat. No. 6,256,057 the array is positioned such that its column-vectors (a column being a group of the array's columns) are parallel to the flight direction. Forward motion compensation (FMC) is provided electronically on-chip (in the detector focal plane array) by the transferring of charge from a pixel to the next adjacent pixel in the direction of flight during the sensor's exposure time (also called “integration time”). The charge transfer rate is determined separately for each column (or for the whole array as in U.S. Pat. No. 6,256,057 where a slit is moved in parallel to the columns direction), depending on its individual distance (range) from the captured scenery, assuming flat ground. In WO 97/42659 this concept is extended to handle transferring of charge separately for each cell instead of column, a cell being a rectangular group of pixels. In the system of U.S. Pat. No. 5,692,062, digital image correlation between successive frames captured by each column is performed, in order to measure the velocity of the scenery with respect to the array, and the correlation result is used for estimating the average range of each column to the scenery, for the purpose of motion compensation in terrain with large variations. This compensation method requires capturing of three successive frames for each single image, two for the correlation process and one for the final motion-compensated frame. The system of U.S. Pat. No. 5,668,593 uses a 3-axis sightline stepping mechanism for expanding coverage of the area of interest, and it applies a motion compensation technique by means of transferring of charge along columns. U.S. Pat. No. 6,130,705 uses a zoom lens that automatically varies the camera field of view based on passive range measurements obtained from digital image correlation as described above. The field of view is tuned in accordance with prior mission requirements for coverage and resolution.
A significant problem which is characteristic of the prior art reconnaissance systems, particularly said electronically scanning Across-Track and Along-Track scanning methods, is the need for predefining for the aircraft an essentially straight scanning leg (and generally a plurality of such parallel straight legs), and once such a leg is defined, any deviation, particularly a rapid or large deviation, from the predefined leg, is not tolerated, as said systems of the prior art are not capable of maintaining a desired line of sight direction during such a fast and/or large deviation from the predefined leg, resulting in image artifacts such as tearing (dislocation of image lines), smearing (elongation of pixels) or substantial gaps in the image information. This is particularly a significant drawback when carrying out a reconnaissance mission above or close to a hostile area, when the need arises for the aircraft to carry out fast maneuvering to escape enemy detection or targeting. Moreover, sometimes, in order to obtain good imaging of a complicated terrain, such as of a curved canyon, it is best to follow the course of the sharply curved edges of the canyon. However, in most cases the reconnaissance systems of the prior art cannot tolerate carrying out such a sharply curved maneuvering, involving sharp changes in the angles of the line of sight with respect to the photographed scenery.
Another drawback characteristic of the reconnaissance systems of the prior art, for example, U.S. Pat. No. 5,155,597, U.S. Pat. No. 5,692,062, WO 97/42659, and U.S. Pat. No. 6,256,057, is their need to handle vast amounts of data. The systems of the prior art do not enable an easy, selective imaging of small portions of an area of interest. Once operated, the system scans the entire area to which the camera is directed, with essentially no selection of specific portions of the whole possible. Therefore, even for a small area of interest, the systems of the prior art must handle a huge amount of data, i.e., be capable of storing the full image data obtained during the operation of the camera, and transmission of it to the ground (when such an option is desired). The transmission of a huge amount of data to the ground, sometimes in real-time, requires usage of a very wide bandwidth. Another particular problem which evolves from this limitation is the need for distinguishing and decoding a small data of interest within the said full, huge amount of data obtained.
Still another drawback of reconnaissance systems of the prior art, for example, U.S. Pat. No. 5,155,597, U.S. Pat. No. 5,692,062, WO 97/42659, U.S. Pat. No. 6,130,705, and U.S. Pat. No. 6,256,057 is their limited ability to capture images in a wide range of a field of regard. Hereinafter, the term “field of regard” refers to the spatial section within which the camera line of sight can be directed without obscuration. Systems of the prior art sometimes use separate dedicated sensors for different sight directions (e.g. separate sensors for down-looking, side-oblique or forward-oblique). The present invention provides to the aircraft the ability of capturing images, simultaneously from all sensors, of areas forward, backward, sideways and in any other arbitrary direction, and to rapidly switch between these directions.
Yet another drawback of reconnaissance systems of the prior art, for example, U.S. Pat. No. 5,155,597, U.S. Pat. No. 5,668,593, U.S. Pat. No. 5,692,062, WO 97/42659, U.S. Pat. No. 6,130,705, and U.S. Pat. No. 6,256,057 is the use of large-sized two-dimensional sensors' arrays, which becomes a necessity for systems having limited or no control over their line of sight. The present invention enables usage of small or medium-sized, two-dimensional sensors' arrays, by taking advantage of the capability to quickly and accurately move the LOS within a large field of regard, to stably fix the LOS on the ground scenery while capturing an image, and to gather photographic image data by a multitude of small/medium frames rather than one single large frame at a time. A small-sized array would typically be up to 1 megapixels (million pixels), and a medium-sized array would typically be up to 5 megapixels. In contrast, large-sized arrays would typically be up to 50 megapixels and even larger. An important feature of the present invention is that both the small and medium-sized arrays are commercially available as universal sensors' arrays, not designed specifically for reconnaissance applications but rather for commercial applications such as stills and video cameras, and therefore they are widely available from a few vendors at low prices. This sensors' technology also benefits from the enormous investment by vendors in such commercial products due to the demands of the commercial market. In contrast, the large-sized reconnaissance sensors' arrays are uniquely developed by reconnaissance systems manufacturers, are complex due to the need for on-chip motion compensation, are expensive, and are not widely available. The limitations of prior art systems are more acute when the sensor is required to operate at the IR range rather than at the visible range, since the current IR array technology does not provide large-sized IR arrays. Another drawback of large-sized arrays is their lower frame rate with respect to small/medium-sized arrays, due to the large amount of pixels processed for each image.
Some of the prior art systems employ on-chip motion compensation, for example, as described in U.S. Pat. No. 5,155,597, U.S. Pat. No. 5,692,062, and WO 97/42659. Several drawbacks are associated with the on-chip motion compensation concept. On-chip motion compensation is performed by transferring charges from one column/cell to an adjacent column/cell during the integration time at a specified rate. This process of transferring charges induces electronic noises and creates an ambiguity (resulting in smearing or loss of pixels) at the borders between columns/cells and at the edges of the chip, since the required charge transfer rate may be different between adjacent columns/cells. Some of the prior art systems assume flat and horizontal ground for estimating the range from the sensor to each part of the scenery in the captured image (i.e. longer range for the farther portion of the scenery and shorter range for the closer portion), and calculate the motion compensation rate based on simple aircraft velocity and attitude information with respect to the flat ground. When the terrain has large variations this generally results in substantial smearing as shown in example 1 of the present invention. In some cases, the sensor must be oriented during capturing so that its columns are accurately parallel to the flight direction without rotation, whereby any deviation from that orientation will result in further smearing, thus seriously limiting mission planning. The more advanced prior art systems use digital image correlation between successive frames for each cell in the chip, in order to estimate more accurately the range to the scenery for each cell. This process requires three successive image captures for each usable image, thus wasting system duty cycles. The correlation accuracy is limited by smearing of the first two images when photographing a terrain with large variations. Another problem associated with correlation is the large change of aspect angle with respect to the scenery between the two successive images. For example, an aircraft flying at a velocity of 250 m/s at a range of 15 km to the scenery in side oblique, using a chip with 2 Hz frame rate, will have an LOS angular velocity (sometimes called V/R) of 250/15=16.7 milirad/s, resulting in an aspect angle between successive images of 8.3 milirad. For a typical pixel Instantaneous FOV (IFOV) of 30 microrad this means a shift of 277 pixels in the image. Moreover, since the value of V/R is not constant at any time during the mission, especially when the aircraft is maneuvering, the elapsed time between the two successive images will induce an additional error.
Some of the prior art systems employ a step framing method to cover large areas, for example, as described in U.S. Pat. No. 5,668,593. The step framing method does not provide mechanical/optical fixing of the LOS on the scenery during exposure time, and has a limited field of regard. On-chip motion compensation is used, but inaccuracies are induced due to vibrations of the aircraft, and delays in transferring the measurement of the vibrations to the reconnaissance system.
It is therefore an object of the present invention to provide a reconnaissance airborne system capable of tolerating and compensating for very sharp maneuvers of the aircraft and for large terrain variations, while still providing high resolution and reliable images of the area of interest, within a very wide field of regard.
It is still another object of the present invention to provide a reconnaissance system in which the amount of irrelevant data is significantly reduced, therefore reducing work needed for distinguishing relevant data from the fully obtained data, and reducing airborne and ground image storage and communication requirements.
It is still another object of the present invention to enable the defining of very small areas of interest within a large area (i.e., a field of regard), of which images can be obtained.
It is still another object of the present invention to reduce the communication load between the aircraft and a ground base station, when communicating images from the aircraft to the ground.
It is still another object of the present invention to provide an airborne reconnaissance system with the ability to capture images in a wide range of the angle of sight (i.e., a wide field of regard).
It is still another object of the invention to provide a new and efficient manner of obtaining the images required for creating stereoscopic-view images.
It is still another object of the invention to provide the capability of combining in the same reconnaissance mission both manual mode operation and automatic mode operation.
Other objects and advantages of the present invention will become apparent as the description proceeds.